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ABSTRACT 

Maps or graphical browsers of structural knowledge 
allow the user of a hypertext system to navigate from node to node 
through the structure as they portray the links between concepts 
spatially. This study examined the results of using two different 
graphical browsers that provide different amounts of information 
about the structure of the knowledge. The use of graphical browsers 
was compared with that of hotwords embedded in the instructional text 
itself but having no explicit structure. Subjects were 113 
undergraduates for whom verbal Scholastic Aptitude Test scores served 
as a measure of verbal ability. Treatment effect was assessed through 
posttests, and student paths were recorded for each treatment. It was 
not evident from the results that all learners internalized the 
structural knowledge provided by the graphical browsers. Those with 
high prior knowledge showed a greater increase in structural 
knowledge. Use of hotwords resulted in lower achievement, and verbal 
ability was not a factor on most variables. Users of hypertext appear 
to require extended experience to become comfortable and proficient. 
Twelve tables present study findings. (Contains 68 references.) 
(SLD) 
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Background 

Information presented in instruction consists of declarative and procedural 
knowledge as well as structural knowledge - the way declarative knowledge concepts are 
interrelated (Jonassen, 1990). Individuals selectively incorporate this new information into 
their existing cognitive structures and then utilize the information in these frameworks 
when applying the knowledge to new situations. Knowledge of content structure portrayed 
in instructional materials can help the learner in two ways: 1) by organizing information in 
short-term memory to build internal connections among the new information and 2) by 
integrating this information with existing knowledge structures in long-term memory to 
develop internal connections (Mayer, 1984). 

Strategies. In designing instruction, generally an expert's conception of the structure 
of the knowledge is mapped onto the instructional materials, either implicitly in the text 
structure and text elaboration techniques or explicitly through graphic portrayals of the text 
structure. Beissner and her colleagues (1993) list types of implicit strategies: signalling 
(Meyer, 1975), frames and slots (Armbruster & Anderson, 1985), and elaboration theory 
(Reigeluth & Stein, 1983). Explicit strategies that graphically portray the structure of 
knowledge for the learner or ask the learner to construct such a representation include 
semantic mapping, semantic features analysis, structured overviews, graphic organizers, 
spider maps, pattern notes, concept maps, networking, text mapping, schematizing, 
advance organizers, and insertions such as pictures, questions, and verbal elaboration 
(Beissner, Jonassen, & Grabowski, 1993; Gagne, 1986). 

Mapping structural knowledge onto instructional materials is thought to help 
learners assimils^ the new information into their cognitive structures -- adding to existing, 
schema, modifying them, or creating new ones as necessary. Studif.s have shown that a 
student's cognitive structure grows more similar to that of the instructor or the instructional 
materials studied (Shavelson, 1974; Thro, 1978; Naveh-Benjamin, McKeachie & Lin, 1989). 

Hypermedia. Many of these techniques assume the learner is using a linear form of 
instruction, similar to a textbook. The learner is led through a prescribed path and exposed 
to the information in a structured fashion reflecting the expert's model. Some of the newer 
technologies, such as hypertext or hypermedia, offer the ability to give the learner control 
over his/her path through the instructional materials, hopefully discovering the structure of 
the knowledge in the process. Each individual can take a different path, encountering 
different amounts and types of information in different sequences. Hypermedia offers 
both potentials and problems in conveying structural knowledge and assisting learners in 
incorporating this structural knowledge into their own personal frameworks. Such non- 
linear systems offer the ability to present various complex structural representations in a 
domain, to show different points of view, to show diversity among similar examples, and to 
show similarities across different categories (Spiro and Jehng, 1990), They can allow the 
learner to explore alternatives and discover relationships (Heller, 1990). As the learner 
maps out a path through the material, he/she must make choices and become a more active 
learner (Bourne, 1990). 

At the same time, such systems can cause disorientation in the learner, overtax 
cognitive processing capabilities, encourage over-simplification, over-generalizations, and 
over-compartmentalization (Conklin, 1987; Hammond; 1989; Jones, 1987; Spiro and Jehng, 
1990). The learner is required to absorb layers of information, make it personally 
meaningful, and gain a comprehensive picture of the subject (Bourne, 1990). 

Hypermedia's organization of nodes and links perpetuates the designer's 
assumptions about the nature of knowledge (Doland, 1989), but this structure is not as 
explicit as in traditional print materials (Charney, 1987). Some question whether this 
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semantic network reflected in hypertext can be transferred to learners or whether 
hypertext's network is analogous to the mind's semantic network (Duffy & Knuth, 1990; 
Romiszowski, 1990; Whally, 1990; Landow, 1990). 

One of the challenges for instructional designers and learners is how to represent 
the information and connections for learning and information retrieval. Research on text 
structure and comprehension, schema development, metacognition and self-regulation, 
learner control, and text enhancement techniques provide a basis for designing instruction to 
improve acquisition of structural knowledge. Most of this research has dealt with linear, 
text-based instruction. The current study examined some of these findings in light of a non- 
linear computerized database that allows the individual learner control over his/her own 
learning. Horn (1989) describes human memory as associative, sometimes connecting 
information in unpredictable, idiosyncratic ways and other times in hierarchical, structured 
ways. Jonassen (1991) proposes that the node-link structure of hypertext reflects the 
semantic structure of the expert's knowledge. Semantic maps of an expert's or experts' 
structure of a subject could be used to create a node and link structure in hypertext. 

Individual Differences. Research has shown that degree of prior knowledge, i.e., 
existing schemata, influences learning, reading, and inferencing (Clark, 1990; Anderson & 
Pearson, 1984; Roller, 1990; Gay, 1986; Fincher-Kiefer, 1992; Spilich, Vesonder, Chiesi & 
Voss, 1979; Chiesi, Spilich & Voss, 1979). Users may have difficulty tracking the overall 
structure of information in a hypertext system and relating it to their prior knowledge. This 
problem may be exacerbated for students with low prior knowledge by causing 
disorientation and cognitive overload. 

Verbal ability differences may also influence a student's ability to process new 
information in a hypertext system, particularly since students with low verbal ability may 
be inefficient in semantic, syntactical, and pragmatic processing of linguistic information 
(Hunt, 1983). Landow (1989) claims that some of the new technologies change the way 
readers read because determining relationships among the linked concepts, which may not 
be obvious to the user, becomes central to understanding. This requires a new type of 
understanding, especially difficult for poorer readers who tend to strictly follow a text's 
linear presentation and seldom use traditional connective aids such as glossaries and 
introductions in print materials. They are not used to having to create associations among 
concepts on their own. 

Graphical Browsers. New skills and instructional aids may be required to use these 
technologies. Explicit techniques different from those in traditional text can help the learners 
perceive the structure of the knowledge. Spatial learning strategies provide a way of 
graphically portraying structural knowledge for the learner or asking the learner to construct 
such a representation. Breuker (1984) sees them as providing external memory storage and 
explicitly depicting concept interrelationships. They focus attention on relevant information 
and help the learner create internal and external connections among the concepts. 

Maps or graphical browsers of the structural knowledge allow the user of a hypertext 
system to navigate from node to node through the structure while spatially portraying the 
links between concepts (Halasz, 1988). They generally allow the user to directly access any 
node on the map, speed up information access, and prevent disorientation (Tsai, 1989). 
"The learner can use a semantic network as a map of the content knowledge in a curriculum 
and use this map to explore topics that are of interest to him or her at the time the interest 
is present " (Denenberg, 1988, p. 309). Denenberg recommends showing only the structure 
around one node to avoid overwhelming the learner. This avoids some of the problems of 
global maps (Conklin, 1987). 

These graphical browsers are thought to influence information processing by 
facilitating both the organization and integration of new information, similar tb the way 
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signalling techniques such as typographic cues and the placement of certain words highlight 
the text structure (Spyriakis, 1989). Graphical browsers draw the learner's attention to the 
important concepts and how they are related to each other. They can help the learner tie 
the new information to prior knowledge in long term memory, especially when some terms in 
the graphical browser are already familiar to the learner. This tie to already known schema 
may be most useful for learners with high prior knowledge who recognize some concepts and 
can use the browser to help decide where to go next. They benefit most from this learner 
control. 

Graphical browsers can also help the learner organize the new information in short 
term memory by setting up a framework to facilitate encoding. This may be more useful for 
those with low prior knowledge who have little existing framework for the new information 
but could potentially interfere with encoding by those with high prior knowledge who already 
have a well-established framework. A system without such graphical browsers would force 
the learner to generate his/her own framework, creating a more active learner. The 
effectiveness of this navigation technique may be tied to the degree of prior knowledge, both 
of facts and of the relationships between them, i.e., the learner's existing cognitive 
framework. 

Jonassen and Wang (1990, 1992, 1993) and Jonassen (1991) have conducted 
several studies on the use of several types of graphical browsers in hypertext systems to 
acquire structural knowledge. They found no significant increase in structural knowledge 
and, in some cases, an actual decrease in recall. Only those specifically told they would be 
required to create a semantic network after the task showed significantly increased 
structural knowledge. They concluded that merely showing the structural relationships was 
not sufficient to result in encoding. In their studies they used measurement techniques that 
attempted to assess higher level skills and transfer, often achieving a basement effect in the 
scores. Learners were unfamiliar with these testing techniques and were unable to 
complete them. Also, these studies used a very large database, raising the question of how 
much information each learner actually covered. 

Phillips and his colleagues (1992) examined different types of navigational devices in 
a hypertext database and found that those provided with the most minimal navigational 
tools (i.e., hotwords in the text itself) achieved the highest recall. This study investigated 
only recall of facts and concepts, but combined with the Jonassen and Wang studies, it 
raises the question of how much explicit structure is optimal for facilitating information 
processing by learners. The current study examined both higher level structural knowledge 
and declarative knowledge, measured retention over time, measured structural knowledge 
using an assessment technique more understandable to the learners, and compared 
learners with different degrees of prior knowledge in the content area and different degrees 
of verbal ability to determine the optimal degree of explicit structure for different learners. 

The study examined the results of using two different graphical browsers providing 
different amounts of information about the structure of the knowledge to learners who have 
been given a specific, externally imposed objective and a structural knowledge task at the 
beginning of instruction. This was compared to the use of hotwords embedded in the 
instructional text itself showing no explicit structure. These pre-lesson strategies of 
providing objectives and a structural knowledge task were intended to alert the learners to 
th« importance of acquiring structural knowledge and to provide a purpose for their 
browsing. Given that Jonassen and Wang (1990) found many participants were unfamiliar 
with structural knowledge acquisition and integration, hypertext techniques, and methods 
of assessing structural knowledge, the study also provided practice in basic hypertext 
navigation to reduce anxiety and to develop appropriate strategies for navigation and 
integration and familiarity with a graphical browser before the instruction. 

Several authors (vVhalley, 1990; Duchastel, 1990; Spiro and Jehng, 1990) have 
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suggested that hypertext may not be appropriate for many instructional uses. Whalley 
suggests that browsing is the most appropriate use of hypertext. Browsing may not be most 
appropriate for instructional purposes, but it may be appropriate for facilitating discovery 
learning (Bruner, 1960). Duchastel (1990) and Spiro and Jehng (1990) suggest that 
hypertext is inappropriate for highly structured learning tasks. There is little empirical 
research to support these claims. This study examined some of these criticisms of hypertext 
by using it for just such a highly structured content domain that includes facts, concepts, 
and principles. 

Research Questions 

These questions about the educational use of hypertext led to the following research 
questions, 

1. Will students given no explicit graphical representation of relationships among concepts 
(Hotwords group), those given a graphic method for showing the relationships (Links group), 
and those given a graphic method for showing relationships with the relationships labeled 
(Detailed Links groups) have different levels of structural knowledge and perform differently 
on tests of comprehension and recall of facts and concepts? 

2. Is there an interaction between degree of prior knowledge and type of graphical method for 
showing structural knowledge on the learner's immediate and delayed structural knowledge, 
comprehension, and recall of facts and concepts? 

3. Will the graphical method for representing structural knowledge during instruction bring 
the learners' structural knowledge representations closer to that portrayed in the instructional 
materials? 

4. Will the type of graphical method for representing structural knowledge affect the 
retention of facts, concepts, and structural knowledge over time? 

5. Is there an interaction between degree of verbal ability and type of graphical method for 
demonstrating structural knowledge on the learner's immediate and delayed structural 
knowledge, comprehension, and recall of facts and concepts? 

This study examined the use of three, types of navigational techniques representing 
structural knowledge in a hypertext system for their influence on the acquisition of 
declarative and structural knowledge. The effects of prior knowledge, verbal ability, 
amount of the content seen, and time spent using the system on achievement were 
examined. In addition, the change in a student's structural knowledge and its similarity to 
an expert model were assessed. Finally, retention over time was assessed using the same 
measures. 

Methods 

Subjects 

One hundred-thirteen undergraduate volunteers from a large state university 
received extra credit for participating in both sessions of the study. One-hundred forty- 
started the study, but five did not have SAT scores, ten did not return for the second 
session, and twelve did not complete the first session. These subjects were dropped. 
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Treatments 



Subjects were randomly assigned to a system using hotwords, one using a graphical 
browser representing the links between concepts, or one using a graphical browser labeling 
the relationships between concepts. 

A sixty-screen hypertext database on the parts of the heart, circulation, and blood 
pressure based on the materials of Dwyer and Lamberski (1977) was developed for this 
study. This content was selected because it covers a variety of facts, concepts, and processes 
and represents a limited subject domain on which there is general consensus on the 
organization of the content. An overarching structure for the system with each screen 
represented as a node was created. Graphical browsers and hotwords representing all 
nodes linked to each screen were created for the different treatments. Links were 
hierarchical, heterarchical, and following the flow of blood through the heart. 

Each treatment had the same screens with only the structural knowledge 
representation methods differing. The treatments were: 

1. A group which highlighted the terms to be linked that, when clicked, took the learner to a 
related screen without describing how the two screens are related (HOTWORDS). 

2. A graphical browser with links visible but not described but otherwise the same as the 
third treatment (LINKS), and 

3. A graphical browser with links described along the lines (DETAILED LINKS). 

The learner could go to each link in succession from hotspots on the screen. By clicking on all 
the hot buttons, the learner could access the same screens in all three treatments. The 
students were allowed to work through the program at their own pace, accessing whatever 
screens they saw fit. They could exit the program at any rime. 

Instruments 

The students were randomly assigned to the three treatment groups, based on the 
assumption that prior knowledge and verbal ability scores would be equally distributed 
among the groups. 

The following measurements were taken for each subject; 

1) a pretest of prior physiological knowledge, 

2) a pretreatment assessment of structural knowledge, 

3) verbal Scholastic Aptitude Test score, 

4) a posttreatment survey of previous computer use and attitudes toward 
computers, hypertext, and this program, 

5) the number of different screens each subject chose to view, and 

6) the amount of time spent on the treatment. 

7) a 40-question immediate criterion posttest measuring recall of facts and 
concepts and comprehension consisting of three multiple-choice tests and one 
drawing test used to assess recall and comprehension developed by Dwyer 
(1978) (Test), 

8) an alternate form of the 40-question criterion posttest administered 
approximately two weeks after the immediate posttest (Retest), and 

9) a post-treatment assessment of structural knowledge (Tree2) 

From the Test and Retest scores, a change score was calculated for the posttests. From the 
third, the ordered-tree technique, a similarity measure to an "expert" structural knowledge 
representation and a change in structural knowledge were calculated. 

The ordered-tree technique has been used in numerous studies to assess a learners 
cognitive structure (NavehBenjamin and others, 1986; McKeithen and others, 1981; Naveh- 
Benjamin, McKeachie, and Lin, 1987; NavehBenjamin, McKeachie, and Lin, 1989) and is 
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based on a theory of hierarchical cognitive structure that assumes concepts are organized 
hierarchically with single concepts at the lowest level and more abstract categories at the 
higher levels. The technique uses recall theory which describes individuals' tendency to list 
all of one branch of the hierarchy before moving on to the next branch. From a number of 
cued and uncued trials, where subjects put sets of low-level concepts in order, an algorithm 
finds the set of all chunks for each subject and creates representations of the subjects' 
cognitive structures (Naveh-Benjamin & Lin, 1991). It measures amount of organization, 
depth of the hierarchy, similarity to an "expert" organization, and direction of relationships 
between concepts. 

The method was validated by two content experts and checked for reliability through 
a field test. This technique was used to assess structual knowledge both before and again 
two weeks after the treatment in this study to examine a subject's prior structural 
knowledge and then the change after the treatment. The pretreatment and posttreatment 
structural knowledge tasks in chis study were scored using the scoring system for the 
structural knowledge assessment technique developed by Dr. Henry Rueter of the Cognitive 
Science and Machine Intelligence Laboratory at the University of Michigan. 

The difference between the mean pretreatment structural knowledge scores and the 
mean retention posttreatment structural knowledge scores of each group was also used as a 
dependent variable. The pretreatment and posttreatment structural knowledge scores for 
each individual were compared to the score of an expert model of structural knowledge (as 
represented in the structure of the materials). A tree can be very high in structure and have 
very little similarity with the "expert"' tree. Such a tree indicates a highly developed, 
individualistic cognitive framework for this content. Another tree can be highly similar to 
the "expert" tree but only exhibit a moderate degree of structure. Two trees with similar 
degrees of structure can look very different. A change in the degree of organization does not 
necessarily indicate the resulting organization is closer to the "expert" one. Nor does a 
consistent score from pretreatment to posttreatment necessarily mean there is no change. It 
is possible there is a change in the type of organization, but the same degree of organization 
is evident. 

SAT verbal scores were requested to examine the effect of verbal ability on 
performance in the different groups. Prior knowledge was assessed using a general 
physiology test developed by Dwyer for this purpose. Although it. does not directly test 
information on the heart, performance on it has been found to correlate with the 
performance on the criterion-referenced tests (Dwyer, personal correspondence). 

A brief survey to assess attitudes, prior computer experience, prior hypertext 
experience, and use of the hypertext lesson was used immediately after the treatment. 
Followup interviews with a sample of learners were conducted by trained interviewers to 
provide additional qualitative data. 

The audit trails for each student were kept by the computer program. This included 
paths taken and time spent at each point in the program. A count of the number of 
different screens seen was displayed on each screen and recorded by the computer. The 
total time spent on the lesson was also recorded. 

Procedure 

Verbal SAT scores were obtained as a measurement of verbal ability. Before the 
treatment subjects took a basic physiology prior knowledge test developed by Dwyer, used 
the ordered-tree technique of Reitman and Rueter (1980) to measure their existing 
structural knowledge of the concepts in the lesson, and practiced using the navigational 
technique in the treatment. 

For up to an hour, the subjects were provided with four objectives for the lesson and 



ERLC 



795 



8 



then navigated through as many of the sixty screens in the system as they chose to using 
one of the three randomly assigned navigational techniques, and then completed an 
experience and attitude survey and a posttest measuring the ability to construct and label a 
diagram of the heart, knowledge of facts, identification of concepts, and comprehension. A 
sampling of the students were interviewed after the treatment. Path data was recorded by 
the computer for each participant. Approximately two weeks later the students took a 
delayed retention posttest and completed a second ordered-tree to assess structural 
knowledge. 



Data Analysis 

Regression was used to to.st for the effect of the treatments, the influence of verbal 
ability and prior knowledge on recall and retention of facts and concepts, comprehension, 
and structural knowledge, and the interaction between variables and to find the most valid 
predictor variables; those which account for the most variance in the dependent variables. 
This was chosen over a multivariate analysis of variance to preserve the maximum amount 
of data on each independent variable. Multiple regression was used to determine the 
function that best predicts performance on the each of the dependent variables: immediate 
posttest, difference between delayed and immediate posttests, posttreatment structural 
knowledge task, similarity of a subject's structural knowledge to an "expert" representation, 
difference between pre and posttreatment structural knowledge tasks and to test the effect 
of the treatments on each of these variables. 

The variables representing coverage of the hypertext database (Count) and 
persistence (Etime) were also included as predictors. In examining the raw data, these 
variables differed widely among the subjects, so their influence on the dependent variables 
were also sought to reduce their potential confounding effects. 

For each dependent variable, the hypothesized predictors, the potential confounding 
variables, and the interaction of all these variables and the treatment were regressed. 
First, parameter estimates were obtained for all the predictor variables and the interactions 
of these variables with the treatment. Those that were not significant at the .05 level were 
dropped and the equation refit. The resulting function was the fit for that dependent 
variable from the predictor variables given, indicating which variables were significant 
predictors for each dependent variable and what percentage of the total variance of the 
dependent variable they accounted for as a group (i.e., the multiple correlation coefficient). 
When interactions between any of the predictors and the treatment were found, equations 
were fit for each treatment. 

A method developed by the researcher was used to compare the paths the students 
used. This technique was carried out for a sampling of twelve subjects from each treatment. 
It reported information on how often they went forward or backward, where they clicked, 
and whether they moved hierarchically, heterarchically, or following the blood flow. 

The survey and interview data were summarized quantitatively and qualitatively. 
Data on prior computer and hypertext use and perceptions were correlated with the test and 
structural knowledge results. 

Results 

The three treatment groups showed no significant differences on verbal Scholastic 
Aptitutde Test scores or prior knowledge of physiology. Groups means showed a difference 
between the immediate posttest, change from immediate to delayed posttest, posttreatment 
ordered-tree task, and posttreatment tree similarity to an "expert" tree among the groups. 
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Subjects in the Hotwords treatment group consistently scored the lowest except in the 
difference between the pre- and posttreatment trees with those in the Links and Detailed 
Links groups being more similar to each other. The Detailed Links group generally had the 
smallest within group variance except on the difference between the immediate and delayed 
posttests and the difference between the pre- and posttreatment trees (see Tablesl,2, and 
3). On this latter variable there is a marked difference in the Detailed Links group. They 
had a very small difference between the pre- and posttreatment scores (i.e., the post scores 
were only slightly higher). 

Table 1 

Means and Standard Deviations on Immediate Posttest, Delayed Posttest, and Change in 
Posttest Scores by Treatment Group 



Treatment 
Group 



Immediate 
Posttest 



Delayed 
Posttest 



Change from Immediate 
to Delayed Posttest 





M SD 


M 


SD 


M 


SD 


Hotwords 


20.919 (8.207) 


17.838 


(7.984) 


3.081 


(4.159) 


(n=37) 












Links 


23.237 (8.221) 


19.895 


(7.266) 


3.342 


(5.800) 


(n=38) 












Detailed Links23.658 (7.549) 


19.132 


(5.757) 


4.526 


(5.331) 


(n=38) 












All Groups 


22.619 (8.016) 


18.965 


(7.043) 


3.655 


(5.144) 



(n=113) 

Note: The change in posttest scores was computed by subtracting the immediate posttest 
score from the delayed posttest score. A positive number indicates a drop in the score over 
time. 
Table 2 

Means and Standard Deviations on Posttreatment Ordered-Tree Task, Change in 
Structural Knowledge, and Similarity to an "Expert" Ordered-Tree by Treatment Group 



Treatment 
Group 



Ordered-Tree 
Task 



Change from 
Pre to Post 
Ordered-Tree 



Similarity 
to "Expert" 
Ordered-Tree 



M SD 
16.243 (13.621) 
20.158 (14.045) 
19.579 (11.758) 



M SD 
2.243 (14.504) 
3.368 (15.958) 

.058 (11.832) 



M SD 
22.162 (20.581) 
33.289 (22.445) 
27.447 (20.201) 



Hotwords 
(n=37) 
Links 
(n=38) 
Detailed 
Links 
(n=38) 
All Groups 
(n=113) 

Note: The change in ordered-tree scores was computed by subtracting the pretreatment 
ordered-tree score from the posttreatment score. A positive number indicates an increase in 
the amount of structure. 



18.947 (13.315) 1.885 (14.139) 



27.681 (21.404) 




797 



10 



Table 3 

Means and Standard Deviations on Number of Screens Seen and Time in Treatment by 
Treatment Group 



Treatment Group 
seconds) 



Screen Count 



Elapsed Time (in 





M SD 


M SD 


Hotwords 


43.9 (12.7) 


1113 (609) 


(n=37) 






Links 


45.1 (10.5) 


1293 (636) 


(n=38) 






Detailed 


44.9 (12.7) 


1360 (564) 


Links 






(n=38) 






All Groups 


44.7 (11.0) 


1257 (607) 


(n=113) 







While not included in the hypotheses, both time on treatment (Etime).and number 
of different screens seen (Count) were included in the analysis since Elapsed Time 
represents the degree of persistence or use of the navigational tools, depending on how many 
total screens were seen, and Screen Count represents the actual coverage of the content. 
The latter indicates the number of different screens seen out of the sixty possible. Although 
they are highly correlated (r= 61378), both were included. 

Immediate Posttest 

Multiple regression analysis revealed predictors of performance on the immediate 
posttest were treatment .group, prior knowledge, an interaction of number of screens seen 
and treatment group, and an interaction of time spent on the treatment and treatment. 
Approximately sixty percent of the total variance in the criterion variable Test can be 
accounted for by these predictors (R-square=.604873, F=17.52, £=.000i). The predictors of 
this variable were then tested for their significance as main effects and in interaction with 
treatment adjusted for all other variables in the regression (see Table 4). 

A significant main effect was found for Prior Knowledge, Number of Screens Seen, 
Elapsed Time, and Treatment. Because there was a significant interaction between two of 
the predictor variables and the treatment, equations were determined by treatment. The 
equation for each treatment is listed in Table 5. 

Table 4 

Test of Significance of Predictor Variables for the Immed iate Posttest Criterion Variable 



b. .rce 


df 


Sums of Squares 


Mean Square 


F 


P 


Prior 


1 


1567.6573 


1567.6573 


56.78 


.0001* 


Count 


1 


159.0293 


159.0293 


5.76 


.0182* 


Etime 


1 


276.0847 


276.0847 


10.00 


.0021* 


Treatment 


2 


190.4967 


95.2484 


3.45 


.0355* 


Count*Treat 


2 


376.6657 


188.3328 


6.82 


.0017* 


Etime*Treat 


2 


248.8115 


124.4058 


4.51 


.0133* 
*£<.05 



798 



11 



The same predictor variables were not significant in the equations for each of the 
treatments. Prior Knowledge remained significant in a positive direction in all three 
treatment equations. Larger beta weights for the same variable, regardless of sign, 
contribute more to the prediction (Huck, Cormier, & Bounds, 1974, pp. 158), although these 
numbers should not be used for comparison when the predictors are correlated and not 
orthogonal as they are in this study (Darlington, 1968). Beta weights cannot be compared 
across variables due to different measurement scales. For variables such as Elapsed Time, 
the beta weights were relatively small due to the scale being used (i.e., seconds). 

Table 5 

Functions for Immediate Posttest for Each Treatment 



Equation R-square Variable Parameter p 



Treatment 1 


.6656 


Intercept 


-11.790777 


.0326* 


(Hotwords) 


.0001* 


Prior 


1.385386 








Count 


- .178874 


.0636 






Etime 


.008288 


.0001* 


Treatment 2 


.6340 


Intercept 


-13.466263 


.0198* 


(Links) 




Prior 


.768232 


.0018* 






Count 


.340977 


.0009* 






Etime 


.002544 


.1210 


Treatment 3 


.5164 


Intercept 


- 9.501131 


.1094 


(Detailed 




Prior 


.962467 


.0005* 


Links) 




Count 


.236039 


.0275* 




Etime 


.000085 


.9689 



p.<.05 

A plot of the predictor variable Prior Knowledge against the predicted immediate 
posttest score showed the positive direction of its influence. It was a significant predictor in 
all three groups (*p<.G5), but there was no significant interaction between prior knowledge 
and the treatment. The steepest slope, and hence, the strongest influence was evident in 
the Hotwords group, but this difference was not statistically significant. The Links group 
had the lowest parai;;Uer estimate and the least influence from prior knowledge. 

The second significant main effect was the predictor Count which indicated the 
number out of the total sixty screens each subject looked at in the lesson. There was a 
significant interaction between treatment and the number of screens seen. The Links and 
Detailed Links group showed a positive relationship but the Hotwords group showed a 
small, slightly negative relationship. There was very little difference in the parameter 
estimates for the Links and Detailed Links groups. The more screens subjects in both of 
these groups tended to view, the higher their immediate posttest score, whereas, for the 
Hotwords group, the number of screens viewed did not make a difference. 

The Elapsed Time predictor indicated how long each subject spent on the treatment, 
which may be interpreted as a measure of persistence. Only in the Hotwords group was the 
predictor variable Elapsed Time significant at the .05 level. Some of this may have been 
due to collinearity of the predictor variable, in particular the correlation between Screen 
Count and Elapsed Time. An outlier who viewed the lesson for almost 3500 seconds may 
have produced the significant effect from Elapsed Time in this treatment where it is not 
significant in the others. 
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This analysis for the immed^.te posttest indicated a significant main effect for 
treatment at the .0355 level with the Links and Detailed Links scores higher than the 
Hotwords group. Treatment interacted significantly with the number of screens viewed and 
the time spent on the treatment. Prior knowledge was a significant predictor of immediate 
posttest scores in all three treatments, but there was no significant interaction between 
treatment group and prior knowledge. Students with differing degrees of prior knowledge 
did not perform significantly different in the three treatments. The predictor Verbal SAT 
scored used to represent verbal ability did not contribute to prediction of the criterion 
Immediate Posttest. 

Change from Pretest to Posttest 

This variable was formed by subtracting the delayed posttest score from the 
immediate posttest score. A positive number indicates tne loss in score. A negative number 
indicates a higher score on the delayed posttest. Predictors of change in score from 
immediate to delayed posttest were prior knowledge, immediate posttest score, treatment, 
and an interaction between verbal ability and treatment (R-squai2=.386828, F=9.46, 
11=0001). 

The variables Prior Knowledge, Immediate Posttest, and Treatment were 
significant (see Table 6). On this criterion variable, the subjects in the Hotwords group 
(M=3.081) had signficantly less loss in achievement between the immediate and delayed 
posttest followed by the Links group (M=3.342) and then the Detailed Links group 
(M=4.526). The woik of Phillips and others (1992) showed that the subjects given the least 
amount of structure performed better on recall. In the current study, those given the least 
structure performed the worst, but + hey lost less over time. The pattern of loss supports the 
statistical concept of regression to the mean over time. 



Table 6 

Test of Significance of Predictor Variables for the Diffe rence between Immediate and 
Delayed Posttests Criterion Variable 



Source 


df 


Sum of 


Mean 


F 


P 






Squares 


Square 






Prior 


1 


239.49088 


239.49088 


13.84 


.0001* 


Treat 


2 


131.11542 


65.55771 


3.79 


.0258* 


Test 


1 


749.61628 


749.61628 


43.31 


.0001* 


Verbal 


1 


57.14237 


57.14237 


3.30 


.0721 


Verbal*Treat 


2 


136.95793 


68.47896 


3.96 


.0220* 



★ 

£<.05 

Because there was a significant interaction between one of the predictor variables, Verbal 
SAT score, and the treatment, equations were determined for each treatment (see Table 7). 
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Table 7 

Functions for the Difference between Immediate and Delayed Posttests for Each Treatment 



1-J L^ Lid LI \J 11 


R-square 


Variable 


Parameter 


p 


Trpfltmpnt 1 


.2594 


Intercept 


11.291464 


.0230* 


^notworas^ 






- 0547680 


.0207* 






Test 


.342397 


.0022* 






Verbal 


- .006322 


.4070 


Treatment 2 


.3895 


Intercept 


-4.437707 


.4290 


(Links) 




Prior 


- .480898 


.0501 






Test 


.412299 


.0023* 






Verbal 


.020062 


.0539 


Treatment 3 


4899 


Intercept 


-3.736668 


.4115 


(Detailed 




Prior 


- .443104 


.0600 


Links) 




Test 


.550589 


.0001* 






Verbal 


.012112 


.1572 



★ 



£<.05 

Thirteen students actually improved between the immediate and delayed 
posttests (i.e., two in the Hotwords group, eight in the Links group, and three in the 
Detailed Links group). The predictor Prior Knowledge contributed to prediction of the 
change between Immediate and Delayed Posttests. A plot of the predicted difference 
between immediate and delayed posttest scores for each of the groups showed a slight 
negative influence of prior knowledge, although the predictor was significant at the .05 level 
only for the Hotwords group (£=.0207). It approached significance in the Links group 
(p=.0501) and in the Detailed Links group (£=.0600). There was no significant interaction 
b e t W een treatment group and degree of prior knowledge. This variable represents retention 
of knowledge from immediately after the treatment to two weeks. A negative parameter 
estimate here indicates that those with lo ' prior knowledge had a higher difference between 
immediate and delayed posttest scores. The different treatments did not help or hinder 
those with low prior knowledge. 

Score on the immediate posttest was a significant predictor of the difference between 
the immediate and delayed posttest scores in all groups. Those who had higher immediate 
posttest scores tended to decrease more between immediate and delayed posttest than 
those who had lower immediate posttest scores. The different treatments did not interact 
with immediate posttest score to influence retention. The influence was strongest in the 
Detailed Links group followed by the Links group and then the Hotwords group. 

Since verbal ability was not significant as a main effect but interacted with 
treatment, plots of this variable and the predicted value of the criterion variable were drawn 
for each treatment. Only in the Links group did verbal ability approach significance as a 
predictor (p=.0539). It was not a significant predictor in the other groups. This may have 
been due in part to its correlation with prior knowledge which was a significant predictor 
overall, but did not. interact with treatment. 

Posttreatment Structural Knowledge 

Predictors of posttreatment structural knowledge were pretreatment structural 
knowledge, an interaction of prior knowledge and treatment, and an interaction of 
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immediate posttest score and treatment. The multiple correlation coefficient predicted 
about thrity-four percent of the total variance (Rrsquare= 335604, F-5.78, £=.0001). 

There was a significant main effect from the prelreatment ordered-tree task. The 
pretreatment score may reflect both a degree of prior knowledge about the organization of 
the concepts and a comfort-level with the task itself. These task scores represent the degree 
of structure, not the structure's similarity to any standard. A student may have a high 
degree of structure on both the pre- and posttreatment tasks, but the results may represent 
two entirely different structures. Because there was a significant interaction between two of 
the predictor variables, Prior Knowledge and Immediate Posttest, and the treatment, 
equations were determined for each treatment (see Table 9). 
Table 8 

Test of Significance of Predictor Variables for the Pos ttreatment Ordered-Tree Task 
Criterion Variable 



Source df 


Sum of 




Mean 


F 






Squares 


Square 




Treat 


2 


659.9262 


329.9631 


2.63 


.0767 










Treel 


1 


1793.5448 


1793.5448 


14.31 


.0003* 










Prior 


1 


298.4925 


298.4925 


2.38 


Prior*Treat 


2 


812.3316 


406.1658 


3.24 


.0431* 










Test 


1 


157.3070 


157.3070 


1.26 


Test*?reat 


2 


1388.3069 


694.1535 


5.54 



.0052* 



*£<.05 

Table 9 

Functions for the Posttreatment Ordered-Tree Task for Each Treatment 



Equation 


R-square 


Variable 


Parameter 


P 


Treatment 1 


.3005 


Intercept 


-23.910317 


.0688 


(Hotwords) 




Treel 


.300901 


.0939 




Prior 


1.893467 


.0125* 






Test 


- .329488 


.3249 


Treatment 2 


.3897 


Intercept 


3.590528 


.7643 


(Links) 






Treel 




.113976 


.4983 












Prior 


- .501087 


.4137 






Test 


1.136821 


.0009* 


Treatment 3 


.3477 


Intercept 


13.722323 


.1838 


(Detailed 




Treel 


.510772 


.0002* 


Links) 




Prior 


- .088868 


.8707 






Test 


- .086433 


.7554 



*H<.05 
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There was a definite linear relationship between pre- and posttreatment scores with 
those performing higher before the treatment also performing higher after the treatment, but 
the plot of pretreatment ordered-tree task by predicted posttreatment ordered tree task did 
not seem to indicate that a demonstrated understanding and ability to do the task before 
the treatment resulted in much better posttreatment structure. The plot seemed to indicate 
that the overall trend was to perform slightly worse on the posttreatment task, especially at 
the higher range of the pretreatment scores. Although there was not a significant 
interaction between pretreatment and posttreatment orderedtree scores, only in the function 
for the Detailed Links group was pretreatment score a significant predictor of posttreatment 
score. 

The immediate posttest interacted with the treatment but was not a predictor of 
posttreatment ordered-tree score by itself. It was a significant predictor only in the Links 
group. In the Links group there was a definite linear pattern showing a positive 
relationship between immediate posttest score and predicted score on the posttreatment 
ordered-tree task. There was no obvious explanation for this one group performing so 
differently, especially since the Detailed Links group often behaved like the Links group and 
tended to demonstrate similar patterns. It was expected that the immediate posttest 
would have some predictive value for the structural knowledge task as structural knowledge 
is thought to be necessary to answer comprehension questions. Previous studies have not 
shown it to be necessary for factual recall and, in some cases, have found it to be a 
hindrance (Jonassen & Wang, 1992). 

Prior knowledge interacted with treatment but was not a predictor of posttreatment 
ordered-tree score by itself. It was a significant predictor only in the Hotwords group. The 
plot showed a definite positive relationship with higher prior knowledge students doing 
better on the posttreatment ordered-tree task. 



Change in Structural Knowledge 

This variable represents the change in structural knowledge from before the 
treatment to two weeks after the treatment. A positive number indicates an increase in the 
amount of structure- while a negative number indicates a decrease in the amount of 
structure. Predictors of change in structural knowledge were prior knowledge and 
pretreatment structural knowledge (R-square=.321461, F=26.06, £=.0001). There was no 
significant interaction between prior knowledge and the treatment, but there was a positive 
relationship between prior knowledge and change in structural knowledge (£=.0131). Those 
with higher prior knowledge had a significantly greater increase in structural knowledge 
than those with lower prior knowledge. It is important to remember the previously detailed 
difficulties with the posttreatment structural knowledge task when interpreting these 
results. 

Table 10 

Test of Significance of Predictor Variables for the Differ ence between the Pretreatment and 
Posttreatment Ordered-Tree Tasks 

Source df Sum of Mean P P 

Squares Square 

Prior 1 943.3673 943.3673 6.83 .0 

Treel 1 6820.5012 6820.5012 49.38 .01 

*ji<.05 
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A plot of the predicted difference between pretreatment and posttreatment ordered- 
tree task by pretreatment ordeved-tree task showed a negative relationship between these 
variables. This can be interpreted to mean that the better one did on the initial structural 
knowledge task, the less difference there was between the pre and post scores on the task 
to a point. In all the groups, there was a point where the subjects did better on the 
pretreatment task than on the posttreatment task. About forty-three subjects fall in this 
area. This loss of structure was most evident in those with higher scores on the 
pretreatment task. 

Structural Knowledge Compared to an "Expert" 

The pretreatment and posttreatment ordered-trees of each subject were compared to 
each other and to the ordered-tree representing the structure of the information in the lesson 
as created by the ixistructional developer (i.e., the "expert" ordered-tree) using the program 
OTSim 1.0 developed by Dr. Henry Rueter at the Cognitive Science and Machine Intelligence 
Laboratory at the Graduate School of Business at the University of Michigan. This 
technique computes the Hirtle similarity measure between ordered trees. This is the ratio 
of the chunks in common and the total chunks of two ordered trees. A similarity measure of 
1.0 indicates two identical trees and 0 indicates two trees that are maximally dissimilar. 
The original similarity scores were multiplied by ten and, therefore, ranged from 100 for 
identical trees to zero indicating total lack of similarity between two trees. Differences in 
directionality between two trees do not enter into the measure, so this measures loses some 
of the information in each tree, but it is the best available at this time. Average similarity 
measures are shown in Table 11. 

Table 11 

Mean Similarity Measures for Pretreatment and Postt reatment Ordered Trees Compared to 
an "Expert" Tree 

Overall Hotwords Links Detailed 

Links 

h an m sn M Sfi M £I> 

Pretreatment 29.46 (20.09) 27.24 (20.28) 31.76 (19.55) 29.32 (20.56) 

/Expert 

Posttreatment 27.68 (21.40) 22.16 (20.58) 33.29 (22.45) 27.45 (20.20) 

/Expert 

Difference between -1.78 (20.86) -5.08 (19.68) 1.53 (26.30) -1.87 (15.06) 
Pre & Posttreatment 
Similarity Scores 

Pre/Posttreatment 35.74 (28.57) 31.43 (28.30) 37.61 (27.72) 38.08 (29.92) 
Similarity (within subject) 

Note : Pretreatment/Expert is the similarity between a subject's pretreatment ordered-tree 
score and that of the "expert." Posttreatment/Expert is the similarity between a subject's 
posttreatment ordered-tree score and that of the "expert." Pretreatment/Posttreatment 
Similarity is the similarity measure between a subject's pre and posttreatment trees. 
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Difference between Pre & Posttreatment Similarity Scores is found by subtracting the 
pretreatment ordered-tree score from the posttreatment score. 

The results of this analysis must be interpreted cautiously as the ordered- 
tree technique proved difficult and/or frustrating for many students. Eight students 
showed no organization at all in the pretreatment task which may have been due to 
a lack of knowledge or difficulty with the task. The overall mean structure score 
was only 17 before the treatment and 19 after the treatment. This could indicate 
that the task did not discriminate well between students with and without 
structural knowledge. 

Although the statistical analysis indicated no significant difference among 
the treatment groups on the posttreatment-"expert" similarity scores (F=2.61, 
£=.0781), there was a marked difference between the Hotwords group and the Links 
group with the Hotwords group much lower in similarity to the structure represented 
in the lesson (see Tablell). The Hotwords group also demonstrated lower similarity 
in their pretreatment trees and posttreatment trees (M = 31.43). On the average, 
their posttreatment trees were over five points lower than their pretreatment trees. 

The only significant predictor of degree of similarity to the "expert" structure 
of knowledge presented through the links in the hypertext system was an interaction 
of immediate pnsttest and treatment. There was no significant main effect. The 
multiple correlation coefficient between these predictors and the criterion was 
.219811 (F=3.47, £=.0014, see Table 12). 



Table 12 

Test of Significance of Predictor Variables for the Comparison of Posttreatment 
Ordered-Tre* Task and the "Expert" O rdered-Tree 

Source df Sam of Mean F p 

Squares Square 

Treatment 2 1315.6904 657.8452 1.69 .1896 

Prior*Treatment 3 2670.9577 890.3192 2.29 .0830 

Test*Treatment 3 5377.1967 1792.3989 4.60 .0046 



£<.05 



* 



A positive relationship between the predictor, Test, and criterion variable 
was indicated in both the Links and Detailed Links groups, although it was only 
significant in the Links group (£=.0014). 

The interaction of prior knowledge and treatment approached significance , 
sc further analysis was performed. In the Hotwords group, prior knowledge 
approached significance (p=.0562) which would seem to follow the trend for many of 
the dependent variables of prior knowledge being most influential in the Hotwords 
group. It exerted a strong positive influence on the posttreatment similarity score in 
this group. This also seemed to be indicated by the plot of the Detailed Links 
treatment, even though it was not significant in that treatment. In the Links group, 
prior knowledge was not a significant predictor. 



Path Data 
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The computer recorded each screen viewed, what order they were viewed, and how 
long was spent on each screen. To track student usage, a method for classifying each move 
from screen to screen was developed. The method tracked the screen the user was on 
currently, the previous screen, and the next screen chosen to determine: 

1) if the user returned to the previous screen or went on to a new one; 

2) if the user went 

a) up the hierarchy of concepts, 

b) down the hierarchy of concepts, 

c) to a concept across the hierarchy at the same level in the same section, 

d) to a related concept in another section, 

e) to a concept before or after the current one in terms of the flow of blood 
through the heart, 

f) back to tb° objectives screen; 

3) what position in the map was clicked or, in treatment one, where in the text 
the clicked word was located; 

4) how long a user moved from screen to screen before returning t/> the objectives 
screen; 

5) how many screens following the blood flow were linked together; 

6) how many of the sixty screens were viewed; 

7) how many screens were seen more than once; 

8) how many screens were viewed in total; 

9) how long was spent on the treatment; 

10) how many of the instruction screens were viewed; 

11) how long the user spent on the instructions; and 

12) in what order the user chose to start a new section and how often was each 
started. 

This information was gathered for a random sampling of twelve subjects from each 
of the treatment groups and collated for each individual. The data were examined in terms 
of where on the map or in the text the user clicked, what typical moves were made by the 
user, and how much time was spent on the program. 

Number of Screens Viewed 

The program contained sixty screens, other than those in the instructions. In the 
entire sample of one hundred-thirteen subjects, the average number of screens seen by a 
subject was 45 (£D=12), and they perceived that they saw about 75% of the screens 
(£D=20) or 42 screens. They spent an average of 1257 seconds or about 21 minutes on the 
treatments (SD=6Q7 seconds). The groups were largely the same in the average number of 
screens seen. 

Moving Onwards or Returning to the Previous Screen 

Subjects had the choice of going on to a new screen from any point, returning to the 
screen they had just seen, or returning to the Objectives Screen. The groups were largely 
similar in their movements. In general, the movement was onward to a new screen rather 
than returning to the screen just seen. To return to a previous screen, the subject had to 
remember which screen he/she just left and then click on the hotword or button for that 
screen. There was no "return" or "previous" button. Student ratings of feeling lost correlated 
with higher rates of returning to the previous screen (r=.46963, p=.0039) and with a sense 
of confusion instead of clarity (r=.30763, p=.0682). 
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Hierarchical vs. Heter archical vs. Process Movement 



In true hypertext fashion, the user of the program had the ability to link to a wide 
variety of related concepts from each screen. When the program was constructed, it was 
divided into four sections, Characteristics, Parts, Circulation, and Blood Pressure, that were 
all linked to the Objectives Screen. From that screen, the user could go down a hierarchy of 
concepts in each section. The user also had the ability to move back up the hierarchy and 
back to the Objectives Screen. In addition, the user could move to related concepts in the 
same section that were across the hierarchy rather than up or down it. They could also 
move to a related concept in another section. Users of the program were free to follow their 
interests, go to familiar concepts, or tackle new ideas. In the Hotwords group, these ideas 
were all embedded in the text and had to merely be clicked to move to a new screen. The 
user had to infer the relationships from the other words in the text. In the Links group and 
in the Detailed Links group the users clicked buttons in the map at the top of the screen to 
move on to new screens. The Links group merely had the concepts linked together by lines. 
The Detailed Links group had the nature of each relationship between concepts labeled on 
the lines. 

Although the Links group moved in p slightly more hierarchical fashion and chose 
slightlv fewer heterarchical links, the groups were fairly similar in their movements. 
Overall, hierarchical movement was the strategy of choice with moving down the hierarchy 
the most chosen strategy followed by moving across the hierarchy within the section, moving 
up the hierarchy, and moving across sections. The Hotwords group chose to move across the 
hierarchy within sections more than the other groups. This could be a function of a lesser 
awareness of the existence of a hierarchy in the program. With the high percentage of 
movement down the hierarchy, one might expect a similar amount back up, but this was not 
the case. The percentage of upward movement was slightly more than half of the 
percentage of downward movement. 

Users tended to start with the first concept on the Objectives screen and work their 
way through the sections in order. They followed strings of screens ranging from one to over 
seventy screens in a row, but most followed fewer than thirty screens before returning to the 
Objectives screen. 

Another possible screen selection strategy was to follow the flow of the blood through 
the heart as one went from screen to screen. Both the previous location and the ensuing 
location were connected to each screen. Subjects chose this strategy less than 12% of the 
time and most of these choices were following it only for a screen or two. A button or 
hotword on each screen led to the part of the heart immediately before and after the current 
point in the blood flow. Only in the Detailed Links group was this relationship explicitly 
shown, although those in the other two groups could determine this through reading the text 
itself. The patterns of screen selection following the blood flow do not differ greatly among 
the groups. 

Click Locations 

Another explanation for movements though the screens was that students selected the 
screen to go to next, not by the content, but by the placement of the hotword in the text or the 
button on the map. Subjects in the Hotwords group did tend to choose words near the 
beginning of the text. The low numbers at the bottom of the list are partially a function of the 
few screens that had that many words. There is not an indication that they chose only the 
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first word, so placement alone does not appear to explain choice of screens. 

Subjects in the Links and Detailed Links groups used the map at the top of the screen 
to navigate. The word at the top of the map was always the concept one step up in the 
hierarchy. Only concepts directly connected to the current screen were shown. The other 
terms around the current concept button were not necessarily arranged in a hierarchical 
fashion, although related terms were placed symmetrically (e.g., Right Auricle was opposite 
Left Auricle under Auricles). The two groups generally chose locations in the same 
proportions. The first counterclockwise position (upper left corner) was chosen most often, 
followed by the top position which was one step up in the hierarchy. This follows the research 
that people in Western cultures tend to work from the upper left corner of a screen to the lower 
right. The first clockwise position (upper right) was chosen third most often followed by the 
second clockwise position and then the second counterclockwise position. Most screens had at 
least these positions, so these generalizations probably can be made. Anything beyond this is 
largely a function of how often that position appeared in the map. It does appear that 
position was a factor in selection of the next concept. 

Attitudes and Experiences 

Before logging off the computer in session one, each participant completed a survey on 
the computer concerning his/her previous computer experience, perceptions of moving through 
the program, and preferred learning strategies. Most items were 5-point Likert-type scales 
(l=low, 5=high). 

The majority of the students did not feel computers were very important in 
their daily lives and few used computers daily. Students rated the use of hypertext for 
learning as moderately confusing. Students indicated that they found the program somewhat 
overwhelming. This feeling appeared similar across all groups. Related to this overwhelming 
feeling, many students indicated that it was relatively easy to get lost (M=3.37). No one 
group appeared to have this feeling more than any other. On the other hand, the Links group 
appeared to find the program slightly more effective than the other groups. Students felt the 
program was harder than text (M=3.21) with the Hotwords and Detailed Links groups 
especially leaning in that direction. 

Students were asked to comment on the program as an educational strategy. 
Of the 59 who chose to respond, fourteen commented favorably, noting the ability to go back 
for review and self pacing as positive aspects. Several students commented that this new form 
of presenting information takes some getting used to. Many described the experience as 
confusing, frustrating, overwhelming, and long. Some felt it provided too many options and no 
logical order. One student stated, "...it bothered me that it was not easy to see the 
hierarchical structure behind the material. Obviously some material would be more 
important than others in any given lesson. I found that this program seemed to make the 
basic structure unclear." A number of others found the heart content boring or too technical. 
Many felt they missed some screens and easily got lost. In contrast to earlier. statements, 
some found it impossible to use for review because the navigation was very difficult. A few 
expressed a general dislike of working with computers and/or a preference for textbooks. 

Searching Strategies 

Students were asked to estimate how much of the program they had covered. A 
counter indicating how many of the 60 different screens a student had seen was located in the 
upper right corner of each screen. A Pearson's Product Moment Correlation was performed on 
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the number of different screens and the percentage estimated seen. There was only a 
correlation of .35494 (p=.0337) between these two measures, indicating the students did not 
take full advantage of this counter and had no intuitive idea of the size of the program. 

All students indicated they sometimes did not go to screens 'hey already felt they 
knew. Few indicated a willingness to skip such screens often. This may indicate the 
students either had little knowledge of the topic, could not judge what was on the next screen, 
or were unwilling to miss anything. Students indicated they sometimes went to concepts 
familiar to them which may also indicate a hesitancy to miss anything. Interesting, there was 
a positive correlation between amount of prior knowledge and number of screen seen 
(r= 28031, p=.0977). Students with higher prior knowledge had a slight tendency to view 
more screens. Students felt sometimes they viewed screens they had seen before, often 
unwittingly, but also felt they sometimes missed screens that they skipped and then never 
could find again. 

Almost half (10 out of 22) admitted to having some problems with the lesson such as 
confusion due to flipping through so many screens, problems with so many options, and 
trouble finding missed screens to an inability getting back to where one wanted to go. 
Students expressed their difficulties: 

It was kind of confusing. There were several choices each time you went, 
and every time you picked a choice, then it would take you off in a totally 
different direction then you were looking at to begin with. (Hotwords group) 

Sometimes I couldn't get back or had a hard time getting back to where I 
wanted to get back to. The checks were a good idea. Otherwise I would 
have gone back to things two or three times by mistake. (Links group) 

The navigational techniques used and the degree of learner control were new to most 
students, and many had difficulty adapting. The majority of the students (i.e.. 
eighteen out of twenty-two) did not find the navigational technique distracting. 
Some felt it gave the main points to be learned. As one student put it: 

It gave me an idea of what the main points were, some of the more 
important information, so I'd look at those. I'd read through the text and 
then go back to the one I'd have the most trouble with or that would interest 
me the most. Then I'd go back and fill in the gaps and read the stuff I 
wasn't too sure of. (Hotwords group) 

A few found it confusing. Two stated they read the text and then looked at the map, 
but one felt it made her want to move on without reading the text. 

When asked about the strategy they developed to move from screen to 
screen, the majority said they had a strategy, although their definitions of a strategy 
varied. Students based their selections on prior knowledge, on an attempt to cover 
all screens, by relationship, or by locational cues rather than content oriented cues. 

When describing the development of these navigational strategies, most 
tended to start with one technique and stick with it. They tended to use the 
asterisks or checks indicating screens already seen to determine what they had 
covered and went back to the Objectives screen to restart when they hit a dead end 
or got lost. Many never really figured out how to move back through screens. 

When asked if they skipped screens on concepts they already knew, nine 
admitted doing that with three more saying they skipped just a few. Ten never 
skipped any screens. Many said they wanted to look at all the screens. Some said 



they went over those they knew more quickly, and a few began to skip or quickly 
skim screens when they realized they knew some of the information. Many said 
they went to the familiar screens first and then tried to relate the new screens to the 
familiar. Some of this hesitancy to skip screens may have been due to the fact that 
the subject was new to most subjects. 

Twenty of the subjects interviewed said they reread screens by choice, 
especially if they did not understand the topic originally. Twelve of the subjects said 
they reread screens by accident. 

Learner control was a concept fairly new to most of the subjects. They weie 
evenly divided on a preference for learner control or lesson control. Those that liked 
learner control said they liked the choice so they could skip what was known, review 
as necessary, and focus on interests. They felt is involved them more in learning. 
Those that favored lesson control said it provides necessary organization so the 
learner knows what is important, doesn't waste energy in organizing the content, 
and provides a logical order for the information. Several felt lost navigating on their 
own. Others felt they might miss important information, especially on a subject new 
to them. 

Those in the Detailed Links group were asked about their use of the labels 
in the graphical browser. Five of the seven said they read the labels, although most 
said they did not do it all of the time. If this is the case, subjects in this group and 
subjects in the Links group were often performing in the same fashion, thereby 
minimizing the effects of the two treatments. 

At the beginning subjects had been asked to think of the lesson as a new 
way of learning. They were asked to list the good and bad points of this way of 
learning. Good points included 1) it was different and interesting; 2) it provided 
many choices and the chance to skip or review as desired; 3) it provided learner 
control to choose when and in what order screens were read; 4) it interactively 
involved the learner and was less boring; 5) it gave the structure of the information 
in the map; 6) it included useful diagrams; 7) there was a small amount of 
information on each screen; and 8) it was less time consuming than reading. Some 
of these strengths were weaknesses to other subjects. Negative aspects of this way 
of learning mentioned included: 1) it allowed a lot of skipping around; 2) navigation 
was hard, often confusing, and difficult to move back and forth; 3) it did not provide 
a concrete order for the information or emphasize the most important information; 4) 
it is difficult for those with little background knowledge; 5) use of the computer itself 
was distract ng; 6) it was possible to go through it quickly and not learn the 
information; 7) it was hard to keep track of what was covered; 8) it was not an 
interesting topic; 9) the amount of information was overwhelming; and 10) not 
everyone has access to a computer. Overall, students felt there were possibilities for 
this type of learning but it was confusing and did not provide enough structure for 
the information. 

Limitations of the Study 

Testing was a threat in this study. The immediate and delayed posttests 
were alternate versions of the same test possibly causing some improvement in the 
delayed test due to the subjects' experience with the immediate posttest. Another 
threat to internal validity was the method for assessing structural knowledge. Most 
students are not familiar with such assessment methods and have trouble carrying 
them out. Mean scores for the task were only 17 out of 44 (£12=13) before the 
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treatment and 19 (SH=13) two weeks after the treatment. Some students produced 
trees with no or very low structure either before or after the treatment. 

A low correlation was found between the posttreatment ordered-tree 
measuring the structure of a learner's content information and the immediate and 
delayed criterion -referenced tests (r=.33156 for the immediate posttest, r=.36401 for 
the delayed posttest). This is another possible indication that the orderedtree task 
did not differentiate well between the subjects. The question of whether focusing 
on the concepts and their relationships helps or hinders recall of facts has emerged, 
in the research. Jonassen and Wang (1990) showed that stressing this type of 
learning may actually hinder factual recall. In the present study, the students did 
not do worse on the posttests than in similar studies with print materials, but the 
structural knowledge tasks did not show a large acquisition of this type of 
knowledge. 

The comparisons to an "expert" structure were conducted using the structure 
of the content in the lesson. This "expert" tree was constructed by the instructional 
developer and should not be considered that of an expert in the field, but rather that 
represented in the lesson. It was verified by a high school biology teacher and a 
nurse for accuracy. The similarity ratings must be interpreted with caution as there 
aro numerous valid ways to structure this information. Each individual may go 
through the lesson and construct different personal structures. 

The ordered-tree technique also may have influenced the study by alerting 
subjects to important concepts. As these terms were not all that wss evaluated on 
the posttest, they may have provided a false sense of security in selecting what to 
study. Many seemed to put little effort into it at the time of the delayed posttest. 

Experimental mortality was a problem, with twelve students not completing 
the first session, ten not returning to the second, and five dropped due to lack of SAT 
scores. Even with this problem, the groups ended up with almost equal numbers, 
although it is not known if the students lost were randomly distributed across all 
independent variables. 

Generalizability or external validity threats were also present in this study, 
most notably the fact that volunteers who did this for extra credit, not as part of 
class content, were used. 

Another problem was the length of treatment. The idea was to work within 
a constrained subject domain so that all the subjects had the possibility of going 
through all the material. Students not interested in a scientific topic may have 
found even the time spent boring. Twelve subjects chose to virw only one or two 
screens before exiting. At the same time, the experience with the different types of 
structural knowledge representations may not have been enough to see an effect, nor 
were the students actively involved in generating the structural frameworks. 

Multiple regression was used for exploring the data in this study. For the 
equation estimated for the immediate posttest the multiple correlation coefficient 
was fairly high, accounting for sixty percent of the variance in the dependent 
variable, but for the other equations, the multiple correlation coefficient was 
between thirty and thirty-five, accounting for only that percentage of the variance in 
the dependent variable. The low multiple correlation coefficients could be due to 
omitting relevant predictor variables, including irrelevant variables, and/or using 
variables in an incorrect form when the relationships are non-linear. All are possible 
in this study with the most likely being omission of relevant variables. Other 
individual cognitive differences, experience with hypertext, risk-taking behavior, and 
motivational factors might be likely predictors of performance. 
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Conclusions and Recommendations 

This study suggests that users of hypertext require extended experience with 
such a system to become comfortable and proficient using it. Graphical browsers 
may provide a degree of structure for the user, K ut it was not evident from the 
results that all learners internalized this structural knowledge. Those with high 
prior knowledge did better on most variables and showed a greater increase in 
structural knowledge. The use of hotwords rather than a graphical browser resulted 
in lower achievement on the criterion test dependent measures and /as a special 
problem for those with lower prior knowledge. Verbal ability was not a factor on 
most of the variables. Further research incorporating extended hypertext use may 
help uo determine the best methods for aiding learners to acquire knowledge in a 
hypertext system. 

Effect of Using Different Navigational Techniques 

On both the immediate posttest and the change between immediate and delayed 
posttests treatment was a significant predictor. The Hotwords group was forced to generate 
their own framework for the knowledge because the words were embedded in the text, not 
arranged relationally. It was this group that consistently performed the lowest on all 
dependent measures. This group's performance was also the most influenced by degree of 
prior knowledge, most notably on the structural knowledge dependent variables. The Links 
and Detailed Links groups who used the two different forms of the graphical browser 
performed similarly on several of the dependent measures, perhaps due to the members of 
the Detailed Links group who did not consistently read the labels in the graphical browser. 

Additional individual subject variables interacted with the treatments, indicating 
that no one treatment was best for all subjects. The contribution of these variables differed 
on the various dependent measures. 

Influence of Prior Knowledge 

Prior knowledge was a significant predictor of immediate posttest score, the 
difference between delayed and immediate posttest scores, and the difference between 
pretreatment and posttreatment ordered-tree task scores. It was not a significant predictor 
of the posttreatment ordered-tree task score itself. 

Prior knowledge did not interact significantly with the treatment on any of the 
dependent measures except the posttreatment ordered-tree task. On this variable it was 
significant only in the Hotwords group. ..It was significant only in the Hotwords group for 
dependent variables representing the difference between posttest scores. This may indicate 
that learning using the Hotwords treatment which was less obvious in its portrayal of the 
content structure was more difficult for students with lower prior knowledge as evidenced by 
several of the dependent measures. The treatments that provided more structural 
knowledge support appeared to be less influenced by degree of prior knowledge. This study 
suggests further research on methods for compensating for low prior knowledge is 
warranted. 

In an examination of the path information, students with higher prior knowledge 
tended to view more of the total number of screens. This may reflect a greater interest in 
the topic or the general unwillingness to skip screens, even if the information was known. 
Those with lower prior knowledge may have experienced more frustration and exited the 
lesson more quickly. Research on learner control has shown that learners often do not know 
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what is best for their own learning, and, even if they do, may not act on this knowledge 
(Milheim & Martin, 1991; Jonassen, 1986). This is a special problem for those with low 
prior knowledge (Steinberg, 1977; Milheim & Martin, 1991). Even college students have 
difficulty with this self-assessment (Garhart & Hannafin, 1986). 

In the interviews, less than half said they skipped any of the screens they already 
knew and of those, most skipped only a few. More common was the tendency to look at a 
screen and go over it quickly if the content was already known. A number of students 
indicated they used their prior knowledge to begin with familiar screens and then tie new 
screens to this known information. 

Comparisons of subjects' and the "expert" ordered-trees showed no significant 
influence of prior knowledge, although it approached significance in the Hotwords group and 
appeared in a general positive direction in the Detailed Links group. 

Influence of Verbal Ability 

Verbal ability was not a major influence on most of the dependent measures, 
possibly due to its correlation with the independent variable Prior Knowledge. Only in the 
difference between the immediate and delayed posttests did it prove significant, and then 
only in interaction with the treatment. 

Influence of Time on Ti jatment and Number of Screens Seen 

Time spent on treatment and number of screens seen contributed to the variance in 
scores on the immediate posttest only. Treatment group interacted with both variables so 
that subjects in the three groups performed differently on the immediate posttest as 
influenced by these two variables. Number of screens seen contributed significantly only in 
the Links and Detailed Links groups, approaching significance in the Hotwords group, while 
time spent on treatment contributed significantly only in the Hotwords group. 

Many subjects did not view all the screens due to fatigue or boredom, a choice of 
degree of effort to invest, or accidentally missed screens and an inability to find them again. 
Subjects viewing significantly under the sixty screens may not have covered all the 
information evaluated in the posttest. The number of screens seen made a difference on 
posttest scores for the Links and Detailed Links group, but not for the Hotwords group 
where there was a very slight negative influence from the number of screens seen. 

Time spent could represent either persistence in finding all the screens and/or 
amount of time spent reading the screens. The latter may have oeen important only in the 
Hotwords treatment because more persistence may have been required to actually see all 
the screens. The screens covered were marked with asterisks within the text itself, but this 
indication may not have been as obvious to the subjects as the checkmarks used in the 
graphical browsers. It was surprising that those at the low end of the time variable did as 
well as they did on the test, considering they spent such a small amount of time viewing the 
screens. Some of the elapsed time was spent on navigation, as a number of students spent 
the last few minutes looking for screens they had missed rather than reading new material. 

The effect of these two variables was not evident in the change from immediate to 
delayed posttest. Simply viewing the more screens quickly might influence immediate recall 
of the information, but not allow for integration of this information into long term memory. 
Viewing more screens did not translate into a proportionally long time spent on the 
treatment. 

Changes in Structural Knowledge 
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The posttreatment structural knowledge scores in this study did not rise from the 
pretreatment scores for all subjects nor did they necessarily grow closer to an "expert" 
structure reflected in the lesson. Those with high initial structural knowledge showed the 
highest posttreatment scores, especially in the Detailed Links group. It is conceivable that 
some of these students may have used the pretreatment task as an organizer for the new 
information, particularly because the graphical browsers in this group spatially portrayed 
some of the same terms used in the task and indicated their relationships. 

Those who showed greater pretreatment structural knowledge actually did not gain, 
but lost structural knowledge from the pre to posttreatment task. Some of this may be 
attributed to the statistical phenomenon of regression to the mean. There may also have 
been some problem with degree of effort expended on the posttreatment task. Another 
possible explanation could be that some students who had a structure before the task were 
caused to question that structure after exposure to the structure in the lesson but never 
integrated this new structure into their schema. On the otht.. ,v K>se who did better on 

the immediate posttest tended to demonstrate more similarity tc uu. -j?;pert" ordered-tree, 
especially in the Links and Detailed Links groups. This reaffirms the idea that degree of 
structure is not synonymous with similarity to the "expert" structure. Students may have 
lost some of their original structure which was unlike the one in the lesson and moved closer' 
to that portrayed in the lesson, but never reaching the same degree of structure shown 
initially. 

Further research is needed on the relationship between declarative knowledge as 
measured in the immediate and delayed posttests and structural knowledge measured by 
the ordered-tree task. If declarative knowledge is embedded in structural knowledge, one 
would expect a closer correlation in all groups between the immediate posttest and the 
posttreatment ordered-tree and between the delayed posttest and the posttreatment 
ordered-tree. Students who learned the relationships between the concepts in the lesson 
would have also learned facts about those concepts. If, as Jonassen and Wang (1992) 
stated, one interferes with the other, e.g., the acquisition of the relationships between 
concepts interferes with learning details about the concepts, a negative relationship would 
be predicted between scores. In this study, the correlation between the immediate posttest 
and the posttreatment structural knowledge task was only .33 and between the delayed 
posttest and the posttreatment structural knowledge task only .36. 



Problems in Learning from Hypertext 

In general, students in all treatments expressed some confusion in going through the 
treatments. Complete learner control over the pace and path of information seen was new 
to most of the students. Many felt lost and frustrated as they tried to work their way 
through the many options on each screen and formulate their own structure. Only after 
sustained use of such a system for a period of time do users become comfortable with this 
control md develop the strategies needed to make selections and follow subject paths 
(Landow, 1990; Oren, 1990; Remde, Gomez and Landauer, 1987; Marchionini, 1988). 

i'his study's subjects spent an average of only twenty-one minutes on the lesson. 
Many, in fact, spent as much if not more time on the testing. They viewed an average of 
forty-four of the sixty possible screens. This was not a long-term involvement with hypertext 
that would bo needed for familiarization with the technique. Multiple trials with navigation 
through a hypertext system should better prepare the subjects to direct their own learning 
through such a s>^tem. 

Hypertext use also can cause an additional cognitive burden by requiring users 
remember the links jvst made and connect all this information into a coherent whole. Users 
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also had to discover and remember the consequence of following a certain path, where it led, 
and types of information it provided (Heller, 1990). Examination of path data showed a few 
students restarting the same section a number of times, only to terminate this path quickly, 
possibly because he or she decided it was something previously covered. Perhaps some way 
of indicating what percentage of the screens seen under any of the four main section 
headings would help ir this decision making. 

Although the counter in the upper right corner indicated how many of the total 
number of screens had been seen, there was no indication where the remaining screens were 
located. This may not be a problem if the system is used for locating information for a 
specific information need, but students intent on seeing all the screens in an instructional 
situation became frustrated and jumped around searching for a few missing screens. 

This additional cognitive load can distract from the content of the lesson (Mayes, 
Kibby, and Anderson, 1990). Only one student interviewed specifically mentioned 
distraction from the use of the graphical browser, although several mentioned they found the 
browser difficult to use and found keeping track of where they had been a burden. The lack 
of normal reading cues added to the 'difficulty of using such a system, especially for less 
skilled readers (Charney, 1987, Horn, 1990). Only two of the students interviewed stated 
that the graphical browsers consciously indicated the structure of the content to them, and 
just a few said they used that structure to help them decide where to go next. This could 
indicate a lack of understanding of the maps themselves or a lack of attention to these cues 
presented in a fashion unfamiliar to the students. 

The sheer amount of information presented and the number of Visions that needed 
to be made could also have been distracting to the user. A number of those interviewed 
mentioned the difficulty of dealing with so many options and this was listed as a reason for 
preferring lesson control by several. The students were given four major objectives on the 
first screen that formed the basis for the organization of the hypertext screens, but many did 
not appear to use these in their searching. The lack of a clearly evident structured path 
through the screens may have caused students who did not develop a strategy to wander 
aimlessly, mtes relevant material, or form a wrong interpretation (Marchionini, 1988). 
Perhaps with such a short exposure, the subjects did not have the chance to develop the 
needed skills in navigating, absorbing layers of information, integrating new information 
with existing knowledge, gaining a comprehensive picture of the subject, self-management, 
searching, and manipulation of the interface needed for hypertext systems (Bourne, 1990; 
Jonassen, 1989b; Kinzie & Berdel, 1990; Wright, 1990; Marchionini, 1988). 

Motivation could also have been a problem in this study, carried out with volunteer 
subjects who had no stake in learning the content. Their only extrinsic motivation was to 
show up for two sessions to get extra credit points as the content was not related to their 
classwork. The content of the heart was not of great interest to a number of those 
interviewed. With motivation an issue and the lack of focus on the objectives given, some 
students may have merely rambled through the system when given complete learner control 
(Hammond, 1989) and may have exited the program before covering all the material. 

Milheim and Martin (1991) suggested that allowing the learner to control the 
sequence of the content may not be effective when the content has a specific prerequisite 
order. They also suggested if all topics must be covered to successfully complete the program 
or when topics have a hierarchical order, learner selection of content to be covered is not 
effective. The results of this study cannot strongly support or repudiate these claims, but 
the mean scores on the criterion-referenced posttest were equal to or slightly higher than 
those in studies using the linear, print form of the materials when adjusted for test length 
(e.g., Hodes, 1990; Cardinale, 1990). No comparable measures of structural knowledge are 
available. 

This hierarchically structured content may not have been the most appropriate for a 
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hypertext system. Duchastel (1990) and Spiro and Jehng (1990) have suggested that 
hypertext is not appropriate for highly structured learning tasks where students may benefit 
from having that structure explicitly provided to them. Students are not presented the 
information in the most logical sequence, nor are facts and lower level concepts mastered 
before higher level ones (Gagne, Briggs, & Wager, 1988). They are left to discover this 
structure on their own, a task especially difficult for those with no prior knowledge on the 
topic. The whole question of whether or not the structure of the system was ever conveyed to 
the subjects must be raised. The low mean scores on the posttreatment ordered tree task 
(M=17) indicated either a lack of understanding of the content structure, a lack of 
understanding of the task or frustration with the task. 

Students interviewed expressed some difficulty in discovering the major concepts 
first. The system was structured so that the user could move down the hierarchy of 
concepts, but this may not have been obvious to all. Users could cover all the main concepts 
first and then delve into each in more depth as desired. This may be one type of 
recommended hypertext use strategy that requires more explanation or instruction for the 
user. 

Learning the process of the blood flow through the heart was also an aspect of this 
lesson. Such a linear process can be less obvious in a non-linear system unless it is clearly 
delineated. If hypertext is to be used to learn the steps in a process, the steps need to be 
more clearly linked in the proper order. 

Search Strate£ 'es 

Students were asked about their search strategies, both in the survey and in the 
interviews and their path data was analyzed. In the survey, they said they returned to the 
previous screen, saw screens more than once, and missed seeing some screens they wanted 
to get back to "sometimes" to "often" on a five point scale (i.e., never, sometimes, often, most 
of the time, always). Analysis of the path data of a sampling of thirty-six subjects showed 
that they returned to the previous screen about ten percent of the time, going on to a new 
screen eighty-four percent of the time and returning to the main Objectives screen six percent 
of the time. A number of students never appeared to grasp the concept of moving back 
through a path. 

The largest percentage of movement in all groups was down the hierarchy with 
hierarchical movement the most common overall. The highest percentage of options in the 
graphical browsers was down the hierarchy, so this was expected. The Links group 
performed slightly more hierarchical movements and fewer heterarchical movements when 
compared to the other two groups. The Hotwords group tended to move across sections 
more, possibly because the structure was less visible. Only twelve percent of the movements 
across all groups followed the flow of blood and much of this was just one screen to the next, 
not a long string of screens following the blood flow. Students either did not see this as an 
option or chose to follow the more obvious hierarchical framework of the lesson. 

Movement across the hierarchy was largely confined to within a section rather than 
jumping across sections to related concepts. Even without a strict hierarchy indicated in the 
navigational technique, students tended to follow a natural hierarchy, although some 
students indicated in the interviews that they had a hard time perceiving the structure of 
the content. In learning a process such as the flow of blood through the heart, this 
hierarchical movement may be most useful for learning the parts of the heart, but not for 
learning the actual process. If students are to follow this blood flow, it must be more clearly 
delineated in the choices for navigation. 

Where the students clicked on the screen w as also recorded to determine if actual 



placement on the screen was a factor in choosing where to go next rather than the actual 
content itself. Those in the Hotwords treatment tended to choose the top three words in the 
text, although not selecting the top term any more often than the others. The groups with 
the graphical browser tended to select the term in the upper left corner most often, followed 
by the term at the top then the term in the upper right corner. This may indicate the 
tendency to start reading in the upper left corner and proceed to the lower right. In general, 
the terms on the left side of the graphical browser were selected more often that parallel 
terms on the right side. Placement on the map may be just as important in selection of the 
next screen as content. If the hierarchy were more evident in the browser, perhaps this 
would be less the case. 



Implications for Instructional Design 

No one technique proved best for all students. When compared to other studies by 
Dwyer using the same materials in a written form, the scores on the criterion-referenced 
posttests were not greatly improved, although results were better than some dissertation 
studies using print versions of the heart materials (e.g., Hodes, 1990; Cardinale, 1990), nor 
time saved by this instructional method (Dwyer, personal correspondence). No comparisons 
on the structural knowledge task were available. Individual differences, such as prior 
knowledge, accounted for much of the variance in learner achievement. 

In general, the use of this hypertext system was not intuitive for first-time users. 
Students not used to this degree of learner control often felt lost and confused. Many had 
trouble developing a viable strategy for moving through and organizing the information. 
Many used the indicators of screens already seen (i.e., asterisks or checkmarks) and number 
of different screens shown in the upper right corner, but still had trouble moving back 
through a path and selecting from the many options on a screen. 

Several suggestions are made for structuring a hypertext system for learning. 

1. Results suggest that the graphical browser would be a better 
navigational tool than the use of the Hotwords alone, especially for 
those with lower prior knowledge. 

2. Students had difficulty getting back to where they came from. 
Techniques such as path histories of recent screens, an indication of 
the immediate previous screen, a thumbnail of the previous screen, or 
a "return" arrow to go to the previous screenmight be useful. 

3. The graphical browser could be structured to better represent the 
types of relationships. Terms up and down the hierarchy might be 
differentiated from those that jump to other sections or those that 
follow the flow of blood. 

4. A global map or web view showing areas covered and relationships 
beyond the immediate screen and its direct links could be made 
available with the ability to zoom in or out to a global map. 

5. Some way of indicating what percentage of the screens seen under any 
of the four main section headings would help in deciding how much of 
each section has been covered. 

6. Not all students work well with a graphical representation of the 
content structure or prefer this method of navigating, so providing 
somo type of menu listing might be more appropriate for these 
users. It might be possible to give the user a choice, but this decision 
may be difficult for the naive user. 



7. Users with low prior knowledge or those with little experience with 
this type of system may need additional guidance in their initial use 
of the system. Possibly a type of guided tour which would highlight 
the major concepts to be covered first would be useful for an overview 
(Oren, 1990). Such guidance could be available on demand in a menu 
or pop-up window to recommend a path, next step(s), and/or strategy. 
Novice and expert users of the system may prefer different degrees of 
guidance. 

8. Some additional method for getting to missed screen? is needed, 
perhaps integration of information on missed sections or screens into 
an overview map or diagram. 

9. A keyword search capability would be useful for those looking for a 
specific piece of information. 

10. Hypertext on its own is not necessarily instructional. Such a program 
might be best used as part of a large instructional system where 
learners are either given or develop specific objectives for using the 
system. 

11. Highly structured information which might be easiest to learn in a 
specified order may best be taught in another fashion. Processes may 
also benefit from a more structured system with more limited 
hypertext features. 

12. An online notebook could be used for taking notes, copying information 
from the screen, incorporating prior knowledge and information from 
other sources, and/or keeping track of the knowledge structure or path 
covered. 

13. Students intervieweu did not show a desire to provide their own labels 
on the graphical browsers, but this generative activity might be 
included as part of the instructional system. It would also dr?w 
students' attention to the graphical browser and its structural 
knowledge content. 

Suggestions for Future Research 

This was an exploratory analysis designed to determine some of the important 
variables in the use of hypertext systems for learning. Variables such as prior knowledge 
and number of screens seen warrant inclusion in future studies. Several suggestions for 
future research follow. 

1. Improve the use of the ordered-tree technique for assessing structural 
knowledge or explore other possibilities for this assessment. Students 
need greater familiarity with the technique and practice in its use 
beforehand. 

2. Compare the acquisition of structural knowledge in this technique to 
using the print form of the lesson. If there is no gain on any of the 
dependent measures using the hypertext version, there may be little 
reason to invest the effort in its development unless skills such as 
information-seeking and comprehension monitoring in a non-linear 
environment are to be developed. 

3. Using the same basic research design, expand the study for use in a 
real life setting with content appropriate for class use. 

4. Further explore the use of different types of graphical browsers. 

5. -Incorporate further investigation of search strategies both through 
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qualitative data and analysis of path data in long-term studies. 
Comparisons of novice and expert users could help describe the 
development of usable strategies. 

6. Assess motivational and attitudinal issues in the use of these systems 
as well as other individual differences such as risk-taking behavior, 
field articulation, and metacognitive ability. Look at specific aspects 
of reading ability that may affect hypertext use. 

7. Look at the relationship between structural knowledge, factual 
knowledge, and problem solving. 

.8. Study the use of hypertext systems for learning in younger students 

and the cognitive demands of such a systems. 
9. Investigate other navigational methods for using hypertext systems 
and their relationship to structural knowledge. Study how these 
systems can replicate or enlace traditional reading cues. 
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